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Highlights  of  GAO-09-403T,  a  testimony 
before  the  Subcommittee  on  Strategic 
Forces,  Committee  on  Armed  Services, 
House  of  Representatives 


Why  GAO  Did  This  Study 

The  Missile  Defense  Agency  (MDA) 
has  spent  about  $56  billion  and  will 
spend  about  $50  billion  more 
through  2013  to  develop  a  Ballistic 
Missile  Defense  System  (BMDS). 
This  testimony  is  based  on  two 
reviews  GAO  was  directed  to 
conduct  in  2008.  In  addition  to  our 
annual  review  assessing  the  annual 
cost,  testing,  schedule,  and 
performance  progress  MDA  made 
in  developing  BMDS,  we  have  also 
reported  on  MDA’s  targets 
program.  In  this  testimony  we 
discuss  (1)  the  productivity  of 
MDA’s  recent  test  program,  (2)  the 
consequences  of  the  testing 
shortfalls,  and  (3)  key  factors  that 
should  be  considered  as  MDA 
revises  its  approach  to  testing. 

GAO  assessed  contractor  cost, 
schedule,  and  performance;  tests 
completed;  and  the  assets  fielded 
during  2008.  GAO  also  reviewed 
pertinent  sections  of  the  U.S.  Code, 
acquisition  policy,  and  the 
activities  of  a  new  missile  defense 
board. 


What  GAO  Recommends 


We  have  previously  made 
recommendations  to  improve  the 
MDA’s  testing  and  targets  programs 
that  include  establishing  a  revised 
business  case  for  providing  targets 
for  a  robust  flight  test  program  as 
well  as  adding  sufficient  scope  to 
tests  to  enable  an  assessment  of 
the  BMDS’  suitability  and 
effectiveness,  but  MDA  only 
partially  agreed.  We  also  have  a 
draft  report  that  is  currently  with 
DOD  for  comment  that  includes 
additional  recommendations 
regarding  testing. 

View  GAO-09-403T  or  key  components. 

For  more  information,  contact  Paul  Francis, 
202-512-4841,  Francisp@gao.gov. 


DEFENSE  ACQUISITIONS 

Charting  a  Course  for  Improved  Missile  Defense 
Testing 


What  GAO  Found 

The  scale,  complexity,  cost  and  safety  associated  with  testing  the  missile 
defense  system  constitute  a  unique  challenge  for  MDA,  test  agencies  and  other 
oversight  organizations.  This  challenge  is  heightened  by  the  fact  that  missile 
defense  assets  are  developed,  produced,  and  fielded  concurrently.  Overall, 
during  fiscal  year  2008,  testing  has  been  less  productive  than  planned.  While 
MDA  completed  several  key  tests  that  demonstrated  enhanced  performance 
of  BMDS,  all  elements  of  the  system  had  test  delays  and  shortfalls,  in  part  due 
to  problems  with  the  availability  and  performance  of  target  missiles.  GMD  in 
particular  was  unable  to  conduct  either  of  its  two  planned  intercept  attempts 
in  fiscal  year  2008.  While  it  did  subsequently  conduct  one  in  December  2008,  it 
was  not  able  to  achieve  all  primary  objectives  because  the  target  failed  to 
release  its  countermeasures.  As  a  result,  aspects  of  the  fielded  ground- 
launched  kill  vehicles  may  not  be  demonstrated  since  no  more  flight  tests 
have  been  approved.  Target  missiles  continue  as  a  persistent  problem  in  fiscal 
year  2008  as  poor  target  performance  caused  several  tests  to  either  fail  in  part 
or  in  whole. 

Testing  shortfalls  have  had  several  consequences.  First,  they  have  delayed  the 
validation  of  models  and  simulations,  which  are  needed  to  assess  the  system’s 
overall  performance.  As  a  result,  the  performance  of  the  fielded  BMDS  as  a 
whole  cannot  yet  be  determined.  Second,  the  production  and  fielding  of  assets 
has  continued  and  in  some  cases  has  gotten  ahead  of  testing.  For  example, 
enhanced  Exoatmospheric  Kill  Vehicles  will  now  be  produced  and  delivered 
before  they  are  flight  tested.  Third,  MDA  has  relied  on  a  reduced  basis — 
fewer  test,  model,  and  simulation  results — to  declare  capabilities  as 
operational  in  the  field. 

MDA  has  undertaken  a  three-phase  review  of  the  entire  BMDS  test  program 
that  involves  identifying  critical  variables  that  have  not  been  proven  to  date, 
determining  what  test  scenarios  are  needed  to  collect  the  data,  and  developing 
an  affordable,  prioritized  schedule  of  flight  and  ground  tests.  This  review,  as 
long  as  it  continues  to  involve  test  and  evaluation  organizations,  appears  to 
offer  a  sound  approach  for  closing  the  gaps  that  exist  between  testing, 
modeling,  and  simulation.  Critical  to  being  able  to  implement  the  approach 
will  be  addressing  the  factors  that  have  limited  the  productivity  of  the  current 
test  approach,  such  as  the  availability  and  performance  of  targets.  An 
additional  consideration  in  a  new  testing  approach  must  be  to  ensure  that 
assets  are  sufficiently  tested  before  they  are  produced  and  fielded.  An 
important  consideration  in  this  regard  is  for  modeling,  simulation,  and  testing 
events  to  be  re-synchronized  so  that  they  properly  inform  decisions  on 
producing,  fielding,  and  declaring  assets  operational.  Contingency  plans 
could  then  be  formed  for  adjusting  the  pace  of  these  decisions  should 
shortfalls  occur  in  modeling,  simulation,  or  testing.  Because  MDA  has 
indicated  implementation  will  take  time,  managing  the  transition  may  need  to 
include  reassessing  the  ambitious  fiscal  year  2009  test  plan.  In  the  mean  time, 
MDA  will  have  to  be  prudent  in  making  decisions  to  produce  and  field  assets. 

_ United  States  Government  Accountability  Office 


Madame  Chairman  and  Members  of  the  Subcommittee: 

I  am  pleased  to  be  here  today  to  discuss  the  future  of  the  Missile  Defense 
Agency’s  (MDA’s)  testing  program. 

MDA  has  been  charged  with  developing  and  fielding  the  Ballistic  Missile 
Defense  System  (BMDS),  a  system  expected  to  be  capable  of  defending 
the  United  States,  deployed  troops,  friends,  and  allies  against  ballistic 
missiles  of  all  ranges  in  all  phases  of  flight.  In  fulfilling  this  charge,  MDA 
placed  an  initial  set  of  missile  defense  components  in  the  field  in 
December  2005. 

The  National  Defense  Authorization  Acts  for  fiscal  years  2002,  2007  and 
2008  mandated  that  we  prepare  annual  assessments  of  MDA’s  ongoing 
cost,  schedule,  testing,  and  performance  progress.  In  March  2009,  we  plan 
to  issue  our  report  covering  MDA’s  progress  toward  achieving  its  goals 
during  fiscal  year  2008  as  well  as  its  efforts  to  improve  transparency, 
accountability,  and  oversight.  Additionally,  in  September  2008,  we  issued  a 
report  on  MDA’s  Target  Program.  My  statement  today  will  focus  on  the 
testing-related  issues  covered  in  both  reports.  We  conducted  these 
performance  audits  from  February  2008  to  February  2009  in  accordance 
with  generally  accepted  government  auditing  standards.  Those  standards 
require  that  we  plan  and  perform  the  audit  to  obtain  sufficient,  appropriate 
evidence  to  provide  a  reasonable  basis  for  our  findings  and  conclusions 
based  on  our  audit  objectives.  We  believe  that  the  evidence  obtained 
provides  a  reasonable  basis  for  our  findings  and  conclusions  based  on  our 
audit  objectives. 


Background 


The  Missile  Defense  Agency’s  mission  is  to  develop  an  integrated  and 
layered  BMDS  to  defend  the  United  States,  its  deployed  forces,  allies,  and 
friends.  In  order  to  meet  this  mission,  MDA  is  developing  a  highly  complex 
system  of  systems — land,  sea  and  space  based  sensors,  interceptors  and 
battle  management.  Since  its  initiation  in  2002,  MDA  has  been  given  a 
significant  amount  of  flexibility  in  executing  the  development  and  fielding 
of  the  BMDS.  To  enable  MDA  to  field  and  enhance  a  missile  defense 
system  quickly,  the  Secretary  of  Defense  in  2002  delayed  the  entry  of  the 
BMDS  program  into  the  Department  of  Defense’s  traditional  acquisition 
process  until  a  mature  capability  was  ready  to  be  handed  over  to  a  military 
service  for  production  and  operation.  Therefore,  the  program  concurrently 
develops,  tests  and  fields  assets.  This  approach  helped  MDA  rapidly 
deploy  an  initial  capability.  On  the  other  hand,  because  MDA  can  field 
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assets  before  all  testing  is  completed,  it  has  fielded  some  assets  whose 
capability  is  uncertain. 

Because  MDA  develops  and  fields  assets  continuously,  it  combines 
developmental  testing  with  operational  testing.  In  general,  developmental 
testing  is  aimed  at  determining  whether  the  system  design  will  satisfy  the 
desired  capabilities;  operational  testing  determines  whether  the  system  is 
effective,  survivable,  and  suitable  in  the  hands  of  the  user.  MDA  conducts 
testing  both  on  the  ground  and  in  flight.  The  most  complex  of  these  is  an 
end-to-end  flight  test  that  involves  a  test  of  all  phases  of  an  engagement 
including  detecting,  tracking  and  destroying  a  target  with  an  interceptor 
missile.  An  end-to-end  intercept  involves  more  than  one  MDA  element. 

For  example,  a  recent  intercept  test  involved  a  target  flown  out  of  Kodiak, 
Alaska,  tracked  by  the  AN/TPY-2  radar  located  in  Alaska,  and  the  Beale 
upgraded  early  warning  radar  located  in  California,  the  Sea-based  X-band 
radar  and  an  Aegis  radar  located  at  different  points  in  the  Pacific.  All  of 
the  radars  communicated  with  fire  control  centers  in  Alaska  to  guide  an 
interceptor  launched  from  California  to  hit  the  target  over  the  Pacific 
Ocean. 

Due  to  the  complexity,  scale,  safety  constraints,  and  cost  involved,  MDA  is 
unable  to  conduct  a  sufficient  number  of  flight  tests  to  fully  understand 
the  performance  of  the  system.  Therefore,  MDA  utilizes  models  and 
simulations,  anchored  by  flight  tests,  to  understand  both  the 
developmental  and  operational  performance  of  the  system.  To  ensure 
confidence  in  the  accuracy  of  modeling  and  simulation  the  program  goes 
through  a  process  called  accreditation.  The  models  are  validated 
individually  using  flight  and  other  test  data  and  accredited  for  their 
intended  use.  Models  and  simulations  are  used  prior  to  a  flight  test  to 
predict  performance,  the  flight  test  is  then  run  to  gather  data  and  verify  the 
models,  and  then  data  is  analyzed  after  the  flight  and  reconstructed  using 
the  models  and  simulations  to  confirm  their  accuracy. 

MDA  intends  to  group  these  models  into  system-level  representations 
according  to  user  needs.  One  such  grouping  is  the  annual  performance 
assessment,  a  system-level  end-to-end  simulation  that  assesses  the 
performance  of  the  BMDS  configuration  as  it  exists  in  the  field.  The 
performance  assessment  integrates  element-specific  models  into  a 
coherent  representation  of  the  BMDS.  Fundamentally,  performance 
assessments  anchored  by  flight  tests  are  a  comprehensive  means  to  fully 
understand  the  performance  capabilities  and  limitations  of  the  BMDS. 
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In  addition  to  testing,  modeling  and  simulation,  and  performance 
assessments,  MDA  also  has  a  formal  process  for  determining  when  a 
newly  fielded  asset  or  group  of  assets  can  be  declared  operational — that  is, 
cleared  for  use  by  the  warfighter  in  operational  situations.  MDA  uses  a 
variety  of  information  as  a  basis  to  assess  a  new  capability  for  declaration. 
For  example,  MDA  will  define  in  advance  tests,  models,  and  simulations  it 
will  use  to  base  a  specific  decision  on  whether  an  asset  or  capability  can 
be  declared  ready  for  fielding.  Each  capability  designation  so  designated 
represents  upgraded  capacity  to  support  the  overall  function  of  BMDS  in 
its  mission  as  well  as  the  level  of  MDA  confidence  in  the  system’s 
performance. 

To  assess  testing  related  progress  in  fiscal  year  2008,  we  examined  the 
accomplishments  of  ten  BMDS  elements  that  MDA  is  developing  and 
fielding.  Our  work  included  examining  documents  such  as  Program 
Execution  Reviews,  test  plans  and  reports,  and  production  plans.  We  also 
interviewed  officials  within  each  element  program  office  and  within  MDA 
functional  directorates.  In  addition,  we  discussed  each  element’s  test 
program  and  its  results  with  DOD’s  Office  of  the  Director,  Operational 
Test  and  Evaluation.  We  also  interviewed  officials  from  the  Office  of  the 
Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics. 


Test,  Targets  and 
Performance 
Challenges  Continue 
During  Fiscal  Year 
2008 


MDA  continues  to  experience  difficulties  achieving  its  goals  for  testing. 
During  fiscal  year  2008,  while  several  tests  showed  progress  in  individual 
elements  and  some  system  level  capabilities,  all  BMDS  elements 
experienced  test  delays  or  shortfalls.  Most  were  unable  to  accomplish  all 
objectives  and  performance  challenges  continued  for  many.  Table  1 
summarizes  test  results  and  target  performance  for  the  BMDS  elements 
during  the  year. 
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Table  1:  Fiscal  Year  2008  Test  and  Targets  Issues 

Element 

Tests/Activities 
Conducted  as 
Scheduled 

All  Objectives 
Achieved 

Target  Issues 

Airborne  Laser 

No 

Yes 

N/A 

Aegis  Ballistic  Missile  Defense  (BMD) 

No 

No 

Target  availability  delayed  key  test  from 

2008  until  at  least  third  quarter  fiscal  year 
2009. 

Command,  Control,  Battle  Management  and 
Communications 

No 

No 

N/A 

Ground-based  Midcourse  Defense  (GMD) 

No 

No 

Target  failed  to  release  countermeasures 
during  December  2008  flight  test — FTG-05.a 

Kinetic  Energy  Interceptor 

No 

No 

N/A 

Multiple  Kill  Vehicle  (MKV) 

No 

Nob 

N/A 

Sensors 

No 

No 

Target  failed  to  release  countermeasures 
during  July  2008  testing  (FTX-03). 

Space  Tracking  and  Surveillance  System 

No 

No 

N/A 

Targets  and  Countermeasures 

No 

No 

Flexible  Target  Family  delivery  delayed  and 
experienced  cost  growth. 

Terminal  High  Altitude  Area  Defense 
(THAAD) 

No 

No 

Target  experienced  anomaly  during  a 
September  flight  test  resulting  in  a  no-test. 

Source:  GAO  (presentation);  MDA  (data). 


“This  flight  test  was  originally  scheduled  for  fiscal  year  2008,  but  was  later  executed  in  fiscal  year 
2009. 

bThe  MKV  program  was  able  to  achieve  this  test  objective  in  the  first  quarter  of  fiscal  year  2009. 


Because  of  delays  in  flight  test  and  a  key  ground  test,  MDA  was  unable  to 
achieve  any  of  the  six  knowledge  points  the  MDA  Director  had  scheduled 
for  fiscal  year  2008.  In  May  2007,  the  MDA  Director  established  key 
system-level  and  element-level  knowledge  points,  each  based  on  an  event 
that  was  to  provide  critical  information — or  knowledge — for  a  decision 
requiring  his  approval.  For  example,  two  knowledge  points  that  MDA  had 
to  defer  because  of  testing  problems  were  confirmation  of  a  new  target’s 
performance  and  assessment  of  the  SM-3  Block  1A  missile’s  ability  to 
engage  and  intercept  a  long  range  target. 

GMD  in  particular  continues  to  experience  testing  problems  and  delays. 
Based  on  its  September  2006  plan,  MDA  had  expected  to  conduct  7  GMD 
interceptor  flight  tests  from  the  start  of  fiscal  year  2007  through  the  first 
quarter  of  fiscal  year  2009.  MDA  however  was  only  able  to  conduct  two,  as 
shown  in  figure  1. 
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Figure  1:  GMD  Reduction  in  Flight  Test  from  January  2006  to  March  2010 


Source:  GAO  analysis  of  GMD's  flight  test  and  interceptor  fielding  schedule  as  of  2/25/07  and  updated  as  of  1/31/09. 


GMD  was  unable  to  conduct  either  of  its  planned  intercept  attempts 
during  fiscal  year  2008  -  FTG-04  and  FTG-05.  MDA  first  delayed  and  then 
later  cancelled  the  FTG-04  test  in  May  2008  due  to  a  problem  with  a 
telemetry  component  in  the  interceptor’s  Exoatmospheric  Kill  Vehicle. 
The  cancellation  of  FTG-04  removed  an  important  opportunity  to  obtain 
end-game  performance  data  needed  to  develop  GMD  models  and  to  verify 
the  capability  of  the  fielded  Capability  Enhancement  I  (CE-I)  EKV. 
Moreover,  MDA  planned  to  test  the  CE-I  EKV  against  a  dynamic  target 
scene  with  countermeasures  in  both  the  FTG-04  and  FTG-05  flight  tests. 
However,  since  FTG-04  was  cancelled  and  the  target  failed  to  release  the 
countermeasure  in  FTG-05,  the  fielded  CE-I’s  ability  against 
countermeasures  still  has  not  been  verified.  According  to  MDA  no  more 
CE-I  EKV  flight  tests  have  been  approved. 
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The  test  delays  led  MDA  to  restructure  its  flight  test  plan  for  fiscal  year 
2009,  increasing  the  number  of  tests,  compressing  the  amount  of  time  to 
analyze  and  prepare  for  subsequent  tests,  and  increasing  the  scope  of 
individual  tests.  For  example,  MDA  plans  to  conduct  14  of  18  flight  tests  in 
the  third  and  fourth  quarter  of  fiscal  year  2009.  Past  testing  performance 
raises  questions  about  whether  this  is  realistic.  In  fiscal  year  2008,  MDA 
had  planned  to  conduct  18  flight  tests,  but  it  only  accomplished  10,  and 
delayed  several  flight  tests  into  2009.  In  the  next  GMD  end-to-end  flight 
test — FTG-06  in  fourth  quarter  fiscal  year  2009  to  first  quarter  fiscal  year 
2010  — MDA  is  accepting  a  higher  level  of  risk  than  it  previously  expected 
in  conducting  this  first  test  of  an  enhanced  configuration  of  the  Kill 
Vehicle  called  the  Capability  Enhancement  II  (CE-II)1  because  it  will 
include  several  objectives  that  had  planned  to  be  previously  tested,  but 
have  not  been.  For  example,  the  FTG-06  flight  test  will  be  the  first  GMD 
test  assessing  both  a  CE-II  EKV  and  a  complex  target  scene.  Adding  to  the 
risk,  it  will  be  only  the  second  test  using  a  newly  developed  FTF  LV-2 
target.  Moreover,  MDA  in  January  2008  had  merged  FTG-06  and  FTG-07, 
thereby  eliminating  an  additional  opportunity  to  gather  important 
information  from  an  intercept.  FTG-07  will  instead  be  an  intercept  test  of 
the  two-stage  interceptor  intended  for  the  European  site. 


Poor  Target  Missile  Problems  with  the  reliability  and  availability  of  targets  (which  are 

Performance  Continues  to  themselves  ballistic  missiles)  have  increasingly  affected  BMDS 
Hamper  BMDS  Testing  development  and  testing  since  2006.  As  MDA  recently  acknowledged, 

target  availability  became,  in  some  cases,  a  pacing  item  for  the  overall  test 
program.  As  was  noted  in  Table  1,  problems  with  targets  have  reduced 
testing  of  GMD,  Sensors,  and  THAAD  during  2008. 

Repeated  target  problems  and  test  cancellations  have  particularly  reduced 
opportunities  to  demonstrate  the  ability  of  sensors  to  discriminate  the  real 
target  from  countermeasures.  In  the  mid-course  of  flight,  a  more 
sophisticated  threat  missile  could  use  countermeasures  in  an  attempt  to 
deceive  BMDS  radars  and  interceptor  sensors  as  to  which  is  the  actual 
reentry  vehicle.  In  order  to  improve  the  effectiveness  of  the  BMDS  against 
evolving  threats,  MDA  elements  are  developing  advanced  discrimination 
software  in  their  component’s  sensors  to  distinguish  the  threat  reentry 
vehicle  from  countermeasures  and  debris.  The  cancellation  of  FTG-04  and 


1  The  CE-II  was  intended  to  replace  obsolescent  parts,  but  it  has  demonstrated  improved 
performance. 
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subsequent  target  problems  during  FTX-03  and  FTG-05  prevented 
opportunities  to  gather  data  to  test  how  well  discrimination  software 
performs  in  an  operational  environment.  The  current  fielded  configuration 
of  the  GMD  kill  vehicle  has  not  been  tested  against  countermeasures. 

To  address  the  growing  need  for  more  sophisticated  and  reliable  targets 
for  the  future  BMDS  test  program,  MDA  has  been  developing  a  new  set  of 
targets  called  the  Flexible  Target  Family  (FTF),  which  was  intended  to 
provide  new  short,  medium,  and  long-range  targets  with  ground,  air,  and 
sea  launch  capabilities.  It  was  viewed  as  a  family  in  the  sense  that  the 
different  target  sizes  and  the  variants  within  those  sizes  would  use 
common  components.  MDA  embarked  on  this  major  development  without 
estimating  the  cost  to  develop  the  family  of  target  missiles.  MDA 
proceeded  to  develop  and  even  to  produce  some  FTF  targets  without  a 
sound  business  case  and,  consequently,  their  acquisition  has  not  gone  as 
planned.  The  funds  required  for  the  FTF  were  spent  sooner  than  expected 
and  were  insufficient  for  the  development. 

Development  of  all  FTF  sizes  and  variants  has  been  discontinued  except 
for  the  72-inch  diameter  ground-launched  target,  referred  to  as  the  LV-2. 
With  guidance  from  the  Missile  Defense  Executive  Board,  MDA  is 
currently  conducting  a  comprehensive  review  of  the  targets  program  to 
determine  the  best  acquisition  strategy  for  future  BMDS  targets.  It  is 
expected  to  be  completed  in  mid-2009.  Whether  or  not  MDA  decides  to 
restart  the  acquisition  of  the  52-inch  diameter  targets,  or  other  FTF 
variants,  depends  on  the  results  of  this  review. 

The  process  of  qualifying  FTF  target  components  for  the  LV-2  was  more 
difficult  than  expected.  While  many  of  the  LV-2’s  components  are  found  on 
existing  systems,  their  form,  fit,  function,  and  the  environment  they  must 
fly  in  are  different.  Consequently,  many  critical  components  initially  failed 
shock  and  vibration  testing  and  other  qualification  tests  and  had  to  be 
redesigned.  MDA  has  acknowledged  that  the  component  qualification 
effort  ran  in  parallel  with  design  completion  and  initial  manufacturing.  So 
far,  the  resultant  delays  in  the  LV-2  target  have  had  two  consequences. 
First,  a  planned  test  flight  of  the  LV-2  itself  for  the  Space  Tracking  and 
Space  Surveillance  program  was  delayed  and  instead  its  first  flight  will  be 
as  an  actual  target  for  an  Aegis  BMD  intercept.  Second,  because  the  LV-2 
was  not  ready,  that  Aegis  intercept  test  was  deferred  from  fiscal  year  2008 
to  third  quarter  fiscal  year  2009. 
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Other  Consequences 
of  Less  Productive 
Testing 

Overall  Performance  of 
BMDS  Can  Not  Yet  Be 
Assessed 


In  addition  to  delaying  progress  on  individual  elements,  testing  problems 
have  had  other  consequences  for  BMDS.  Specifically,  the  reduced 
productivity  of  testing  has  delayed  understanding  the  overall  performance 
of  BMDS,  production  and  fielding  have  in  some  cases  gotten  ahead  of 
testing,  and  declarations  of  capabilities  ready  for  fielding  have  been  made 
based  on  fewer  tests  and  less  modeling  and  simulation  than  planned. 

The  overall  performance  of  the  BMDS  cannot  yet  be  assessed  because 
MDA  lacks  a  fully  accredited  end-to-end  model  and  simulation  capability 
and,  according  to  the  BMDS  Operational  Test  Agency,  it  will  not  have  that 
capability  until  2011  at  the  earliest.  The  lack  of  sufficient  flight  test  data 
has  inhibited  the  validation  of  the  models  and  simulations  needed  for  the 
ground  tests  and  the  simulation.  MDA’s  modeling  and  simulation  program 
enables  it  to  assess  the  capabilities  and  limitations  of  how  BMDS  performs 
under  a  wider  variety  of  conditions  than  can  be  accomplished  through  the 
limited  number  of  flight  tests  conducted.  Flight  tests  alone  are  insufficient 
because  they  only  demonstrate  a  single  collection  data  point  of  element 
and  system  performance.  Flight  tests  are,  however,  an  essential  tool  used 
to  both  validate  performance  of  the  BMDS  and  to  anchor  the  models  and 
simulations  to  ensure  they  accurately  reflect  real  performance.  Computer 
models  of  individual  elements  replicate  how  those  elements  function. 
These  models  are  then  aggregated  into  various  combinations  that  simulate 
the  BMDS  engagement  of  enemy  ballistic  missiles. 

Developing  an  end-to-end  system-level  model  and  simulation  has  been 
difficult.  MDA’s  first  effort  to  bring  together  different  element  models  and 
simulations  to  produce  a  fully  accredited,  end-to-end  model  and 
simulation  was  for  the  first  annual  performance  assessment  of  the  fielded 
BMDS  configuration  in  2007.  Performance  Assessment  2007  was 
unsuccessful  primarily  because  of  inadequate  data,  particularly  flight  test 
data,  for  verification  and  validation  to  support  accreditation.  Instead, 
Performance  Assessment  2007  used  several  models  and  simulations  that 
represented  different  aspects  of  the  BMD  system  and  were  not  fully 
integrated.  Consequently,  acting  on  a  joint  recommendation  between  MDA 
and  the  Operational  Test  Agency,  MDA  officials  cancelled  the  2008 
performance  assessment  in  April  2008  because  of  developmental  risks 
associated  with  modeling  and  simulations,  focusing  instead  on  testing  and 
models  for  Performance  Assessment  2009. 

According  to  the  BMDS  Operational  Test  Agency’s  January  2009  Modeling 
and  Simulation  accreditation  report,  confidence  in  MDA’s  Modeling  and 
Simulation  efforts  remains  low  although  progress  was  made  during  the 
year.  Out  of  40  models,  the  BMDS  Operational  Test  Agency  recommended 
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in  January  2009  full  accreditation  for  only  6  models,  partial  accreditation 
for  9  models,  and  no  accreditation  for  25  models.  MDA  is  now  exercising 
stronger  central  leadership  to  provide  guidance  and  resources  as  they 
coordinate  the  development  of  verified  and  validated  models  and 
simulations. 

MDA  intends  to  verify  and  validate  models  and  simulations  by  December 
2009  for  Performance  Assessment  2009.  However,  BMDS  Operational  Test 
Agency  officials  stated  that  there  is  a  high  risk  that  the  performance 
assessment  2009  analysis  will  be  delayed  because  of  remaining  challenges 
and  MDA’s  delayed  progress  in  accreditation.  MDA  does  not  expect  to 
have  a  single  end-to-end  simulation  for  use  in  performance  assessments 
until  2010. 


Production  and  Fielding 
Proceed  Despite  Delays  in 
Testing  and  Assessments 


Testing  problems  have  contributed  to  a  concurrent  development, 
manufacturing  and  fielding  strategy  in  which  assets  are  produced  and 
fielded  before  they  are  fully  demonstrated  through  testing  and  modeling. 
For  example,  although  a  test  of  the  ability  of  the  SM-3  Block  1A  missile  to 
engage  and  intercept  a  long  range  ballistic  target  was  delayed  until  the 
third  quarter  of  fiscal  year  2009,  MDA  purchased  20  of  the  missiles  in  fiscal 
year  2008  ahead  of  schedule. 


While  the  GMD  program  has  only  been  able  to  conduct  two  intercepts 
since  2006  for  assessing  the  fielded  configuration,  the  production  of 
interceptors  has  continued.  From  the  beginning  of  fiscal  year  2007  through 
the  first  quarter  of  fiscal  year  2009,  MDA  planned  to  conduct  7  flight  tests 
and  field  16  new  ground-based  interceptors.  The  plan  included  a  test  that 
would  utilize  two  ground-based  interceptors  against  a  single  target,  known 
as  a  salvo  test.  By  January  2009,  GMD  had  conducted  only  2  flight  tests 
and  dropped  the  salvo  test;  yet  it  fielded  13  ground-based  interceptors. 

Moreover,  the  GMD  program  had  planned  to  conduct  an  intercept  test  to 
assess  the  enhanced  version  of  the  EKV  called  the  Capability 
Enhancement  II  (CE-II)  in  the  first  quarter  of  fiscal  year  2008,  months 
before  emplacing  any  interceptors  with  this  configuration.  However, 
developmental  problems  with  the  new  configuration’s  inertial 
measurement  unit  and  the  target  delayed  the  first  flight  test  with  the  CE-II 
configuration — FTG-06 — until  at  least  fourth  quarter  fiscal  year  2009. 
Despite  these  delays,  emplacements  will  proceed;  MDA  expects  to  have 
emplaced  five  CE-II  interceptors  before  this  flight  test.  More  importantly, 
GMD  projects  that  the  contractor  will  have  manufactured  and  delivered  10 
CE-II  EKVs  before  that  first  flight  test  demonstrates  the  CE-II  capability. 


Page  9 


GAO-09-403T 


This  amounts  to  over  half  of  the  CE-II  EKV  deliveries  that  are  currently 
under  contract. 

Declaration  of  Capabilities 
Proceed  with  Reduced 
Levels  of  Information 


In  fiscal  year  2008,  MDA  declared  it  had  fielded  7  of  17  BMDS  capabilities 
planned  for  2008  (postponing  10).  In  doing  so  MDA  largely  reduced  the 
basis  for  the  declarations  due  in  part  to  test  problems  and  delays. 
Specifically,  MDA  had  intended  to  use  a  GMD  flight  test  that  was 
cancelled,  a  key  ground  test  that  was  delayed  and  a  performance 
assessment  that  was  cancelled.  MDA  had  to  shift  the  basis  of  the  7 
declarations  to  previous  flight  and  ground  tests. 


When  MDA  determines  that  a  capability  can  be  considered  for  operational 
use  it  does  so  through  a  formal  declaration.  MDA  bases  its  declarations  on, 
among  other  things,  a  combination  of  models  and  simulations — such  as 
end-to-end  performance  assessments  (from  missile  launch  to  attempted 
intercept) — and  ground  tests  all  anchored  to  flight  test  data. 


Review  of  BMDS 
Modeling  and  Testing 
Holds  Promise,  but 
Must  Anticipate 
Contingences 


MDA  has  undertaken  a  three-phase  review  of  the  entire  BMDS  modeling, 
simulation,  and  test  program.  According  to  MDA,  the  three  phases  involve 
identifying  critical  variables  that  have  not  been  proven  to  date, 
determining  what  test  scenarios  are  needed  to  collect  the  data,  and 
developing  an  affordable  and  prioritized  schedule  of  flight  and  ground 
tests.  MDA  intends  to  complete  all  three  phases  of  the  review  by  May  2009. 
At  this  point,  our  knowledge  of  the  review  is  limited,  as  we  have  only  had 
an  introductory  briefing  on  it.  Nonetheless,  the  review  appears  to  offer  a 
sound  approach  for  closing  the  gaps  that  exist  between  testing,  modeling, 
and  simulation.  Further,  the  involvement  of  test  and  evaluation 
organizations  is  encouraging. 


While  sound,  the  success  of  this  approach  hinges  on  providing  sufficient 
resources,  ensuring  robustness,  and  anticipating  contingencies.  In 
addition  to  linking  the  critical  modeling  and  simulation  variables  with  test 
events,  the  review  will  have  to  address  the  factors  that  have  limited  the 
productivity  of  the  current  test  approach,  such  as  the  availability  and 
performance  of  targets.  MDA’s  current  approach  to  testing  could  be 
characterized  as  a  just-in-time  approach  to  having  the  test  assets,  such  as 
targets,  ready.  This  left  little  margin  to  solve  issues  that  arise  leading  up  to 
the  tests.  Accordingly,  the  third  phase  of  MDA’s  new  approach — properly 
resourcing  the  tests  with  sufficient  time,  funding  and  reliable  targets — will 
be  key.  MDA  has  indicated  that  its  revision  will  result  in  a  more  robust  test 
plan,  providing  more  margin  to  conduct  the  tests  through,  for  example, 
having  spare  interceptors  and  targets  available. 
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Other  contingencies  that  a  new  approach  to  modeling,  simulation,  and 
testing  should  anticipate  include  unexpected  or  incomplete  test  results, 
and  problems  in  accrediting  the  models  that  are  needed  for  aggregated 
simulations,  such  as  performance  assessments.  An  important 
consideration  in  this  regard  is  for  modeling,  simulation,  and  testing  events 
to  be  re-synchronized  so  that  they  properly  inform  decisions  on  producing, 
fielding,  and  declaring  assets  operational.  Contingency  plans  could  then 
be  formed  for  adjusting  the  pace  of  these  decisions  should  shortfalls  occur 
in  modeling,  simulation,  or  testing. 

MDA  has  indicated  that  this  new  approach  to  testing  will  take  time  to 
implement,  with  partial  implementation  in  fiscal  year  2010  and  full 
implementation  not  occurring  until  fiscal  year  2011.  Therefore,  MDA  must 
manage  the  transition  to  the  new  testing  approach.  In  particular,  the 
ambitious  fiscal  year  2009  flight  test  plan  may  need  to  be  reassessed  with 
the  goal  of  establishing  a  robust  series  of  tests  that  can  withstand  some 
delays  without  causing  wholesale  changes  to  the  test  plan  during  the 
transition.  In  the  mean  time,  MDA  will  have  to  be  prudent  in  making 
decisions  to  produce  and  field  additional  assets. 

Our  annual  report  on  missile  defense  is  in  draft  and  with  DOD  for 
comment.  It  will  be  issued  in  final  by  March  13,  2009.  In  that  report,  we 
are  recommending  additional  steps  to  further  improve  the  transparency, 
accountability,  and  oversight  of  the  missile  defense  program.  Our 
recommendations  include  actions  to  improve  cost  reporting  as  well  as 
testing  and  evaluation.  DOD  is  in  the  process  of  preparing  a  formal 
response  to  the  report  and  its  recommendations. 


Madame  Chairman,  this  concludes  my  statement.  I  would  be  pleased  to 
respond  to  any  questions  you  or  members  of  the  subcommittee  may  have. 
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